78 research outputs found
Bayesian estimation of discretely observed multi-dimensional diffusion processes using guided proposals
Estimation of parameters of a diffusion based on discrete time observations
poses a difficult problem due to the lack of a closed form expression for the
likelihood. From a Bayesian computational perspective it can be casted as a
missing data problem where the diffusion bridges in between discrete-time
observations are missing. The computational problem can then be dealt with
using a Markov-chain Monte-Carlo method known as data-augmentation. If unknown
parameters appear in the diffusion coefficient, direct implementation of
data-augmentation results in a Markov chain that is reducible. Furthermore,
data-augmentation requires efficient sampling of diffusion bridges, which can
be difficult, especially in the multidimensional case.
We present a general framework to deal with with these problems that does not
rely on discretisation. The construction generalises previous approaches and
sheds light on the assumptions necessary to make these approaches work. We
define a random-walk type Metropolis-Hastings sampler for updating diffusion
bridges. Our methods are illustrated using guided proposals for sampling
diffusion bridges. These are Markov processes obtained by adding a guiding term
to the drift of the diffusion. We give general guidelines on the construction
of these proposals and introduce a time change and scaling of the guided
proposal that reduces discretisation error. Numerical examples demonstrate the
performance of our methods
Simulation of elliptic and hypo-elliptic conditional diffusions
Suppose is a multidimensional diffusion process. Assume that at time zero
the state of is fully observed, but at time only linear combinations
of its components are observed. That is, one only observes the vector
for a given matrix . In this paper we show how samples from the conditioned
process can be generated. The main contribution of this paper is to prove that
guided proposals, introduced in Schauer et al. (2017), can be used in a unified
way for both uniformly and hypo-elliptic diffusions, also when is not the
identity matrix. This is illustrated by excellent performance in two
challenging cases: a partially observed twice integrated diffusion with
multiple wells and the partially observed FitzHugh-Nagumo model
Applied Measure Theory for Probabilistic Modeling
Probabilistic programming and statistical computing are vibrant areas in the
development of the Julia programming language, but the underlying
infrastructure dramatically predates recent developments. The goal of
MeasureTheory.jl is to provide Julia with the right vocabulary and tools for
these tasks.
In the package we introduce a well-chosen set of notions from the foundations
of probability together with powerful combinators and transforms, giving a
gentle introduction to the concepts in this article.
The task is foremost achieved by recognizing measure as the central object.
This enables us to develop a proper concept of densities as objects relating
measures with each others. As densities provide local perspective on measures,
they are the key to efficient implementations.
The need to preserve this computationally so important locality leads to the
new notion of locally-dominated measure solving the so-called base measure
problem and making work with densities and distributions in Julia easier and
more flexible
Fast and scalable non-parametric Bayesian inference for Poisson point processes
We study the problem of non-parametric Bayesian estimation of the intensity
function of a Poisson point process. The observations are independent
realisations of a Poisson point process on the interval . We propose two
related approaches. In both approaches we model the intensity function as
piecewise constant on bins forming a partition of the interval . In
the first approach the coefficients of the intensity function are assigned
independent gamma priors, leading to a closed form posterior distribution. On
the theoretical side, we prove that as the posterior
asymptotically concentrates around the "true", data-generating intensity
function at an optimal rate for -H\"older regular intensity functions (). In the second approach we employ a gamma Markov chain prior on the
coefficients of the intensity function. The posterior distribution is no longer
available in closed form, but inference can be performed using a
straightforward version of the Gibbs sampler. Both approaches scale well with
sample size, but the second is much less sensitive to the choice of .
Practical performance of our methods is first demonstrated via synthetic data
examples. We compare our second method with other existing approaches on the UK
coal mining disasters data. Furthermore, we apply it to the US mass shootings
data and Donald Trump's Twitter data.Comment: 45 pages, 22 figure
Bayesian wavelet de-noising with the caravan prior
According to both domain expert knowledge and empirical evidence, wavelet
coefficients of real signals tend to exhibit clustering patterns, in that they
contain connected regions of coefficients of similar magnitude (large or
small). A wavelet de-noising approach that takes into account such a feature of
the signal may in practice outperform other, more vanilla methods, both in
terms of the estimation error and visual appearance of the estimates. Motivated
by this observation, we present a Bayesian approach to wavelet de-noising,
where dependencies between neighbouring wavelet coefficients are a priori
modelled via a Markov chain-based prior, that we term the caravan prior.
Posterior computations in our method are performed via the Gibbs sampler. Using
representative synthetic and real data examples, we conduct a detailed
comparison of our approach with a benchmark empirical Bayes de-noising method
(due to Johnstone and Silverman). We show that the caravan prior fares well and
is therefore a useful addition to the wavelet de-noising toolbox.Comment: 32 pages, 15 figures, 4 table
Nonparametric Bayesian estimation of a H\"older continuous diffusion coefficient
We consider a nonparametric Bayesian approach to estimate the diffusion
coefficient of a stochastic differential equation given discrete time
observations over a fixed time interval. As a prior on the diffusion
coefficient, we employ a histogram-type prior with piecewise constant
realisations on bins forming a partition of the time interval. Specifically,
these constants are realizations of independent inverse Gamma distributed
randoma variables. We justify our approach by deriving the rate at which the
corresponding posterior distribution asymptotically concentrates around the
data-generating diffusion coefficient. This posterior contraction rate turns
out to be optimal for estimation of a H\"older-continuous diffusion coefficient
with smoothness parameter Our approach is straightforward to
implement, as the posterior distributions turn out to be inverse Gamma again,
and leads to good practical results in a wide range of simulation examples.
Finally, we apply our method on exchange rate data sets
Automatic Backward Filtering Forward Guiding for Markov processes and graphical models
We incorporate discrete and continuous time Markov processes as building
blocks into probabilistic graphical models with latent and observed variables.
We introduce the automatic Backward Filtering Forward Guiding (BFFG) paradigm
(Mider et al., 2020) for programmable inference on latent states and model
parameters. Our starting point is a generative model, a forward description of
the probabilistic process dynamics. We backpropagate the information provided
by observations through the model to transform the generative (forward) model
into a pre-conditional model guided by the data. It approximates the actual
conditional model with known likelihood-ratio between the two. The backward
filter and the forward change of measure are suitable to be incorporated into a
probabilistic programming context because they can be formulated as a set of
transformation rules.
The guided generative model can be incorporated in different approaches to
efficiently sample latent states and parameters conditional on observations. We
show applicability in a variety of settings, including Markov chains with
discrete state space, interacting particle systems, state space models,
branching diffusions and Gamma processes
- …